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Abstract 

Background: Research in the field of Empirical Ethics (EE) uses a broad variety of empirical methodologies, such as 
surveys, interviews and observation, developed in disciplines such as sociology, anthropology, and psychology. Whereas 
these empirical disciplines see themselves as purely descriptive, EE also aims at normative reflection. Currently there is 
literature about the quality of empirical research in ethics, but little or no reflection on specific methodological aspects 
that must be considered when conducting /nfe/'G';'5op//na/y empirical ethics. Furthermore, poor methodology in an EE 
study results in misleading ethical analyses, evaluations or recommendations. This not only deprives the study of 
scientific and social value, but also risks ethical misjudgement. 

Discussion: While empirical and normative-ethical research projects have quality criteria in their own right, we focus 
on the specific quality criteria for EE research. We develop a tentative list of quality criteria - a "road map" - tailored to 
interdisciplinary research in EE, to guide assessments of research quality. These quality criteria fall into the categories of 
primary researcli question, tlieoretical frameworl< and methods, relevance, interdisciplinary research practice and research 
ethics and scientific ethos. 

Summary: EE research is an important and innovative development in bioethics. However, a lack of standards 
has led to concerns about and even rejection of EE by various scholars. Our suggested orientation list of criteria, 
presented in the form of reflective questions, cannot be considered definitive, but serves as a tool to provoke 
systematic reflection during the planning and composition of an EE research study. These criteria need to be 
tested in different EE research settings and further refined. 

Keywords: Empirical ethics. Evidence-based ethics. Empirical methodology, Applied bioethics, Interdisciplinarity, 
Methodology, Quality criteria 



Background^ 

Empirical ethics 

For roughly two decades there have been debates in 
bioethics about the question of how to address the 
challenge of best practice in interdisciplinary methodology. 
Empirical research in bioethics, principally using the 
methods of social sciences [1,2]'', has considerably increased 
during this period (e.g. [3]). 

Generally speaking, this debate comes under the label 
of what is known as "empirical ethics" (abbreviated "EE", 
e.g. [4-8]); some authors prefer to talk about "empirically 
informed ethics" or sometimes "evidence-based ethics" (e.g. 
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[9-13]). This field calls for more empirical research, mainly 
from sociology, psychology or anthropology, and/or more 
consideration of empirical research results in normative 
bioethics'^. Whereas empirical disciplines aim to be purely 
descriptive, however, EE has a strong normative objective: 
empirical research in EE is not an end in itself, but a 
required step towards a normative conclusion or statement 
with regard to empirical analysis, leading to a combination 
of empirical research with ethical analysis and argument. 

Research problem 

The widespread use of EE highlights the importance 
question above: what is the best practice for applying 
empirical methodologies in such an interdisciplinary 
setting? This interdisciplinary challenge is stiU not solved, 
and proponents of EE can self-critically assume that the 
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quality of EE stxidies is often unsatisfactory. This problem 
can be tackled by two strategies: either focusing only on 
one particular methodology, or trying to establish stan- 
dards for 'good' EE research. The advantage of the latter 
is obvious: methodologies are highly dependent on theor- 
etical assumptions, and there is no such thing as one true 
theory, neither in empirical research nor in ethics. It 
therefore seems most appropriate to focus on best practice 
instead of perfecting one particular methodology in EE. 

The lack of standards for assessing and safeguarding 
quality is not only a problem for scientific quality per se, 
but also an ethical problem: poor methodology in EE may 
give rise to misleading ethical analyses, evaluations or 
recommendations'^, not only depriving the study of scien- 
tific and social value, but also risking ethical misjudge- 
ment. Improving the quality of EE is therefore an ethical 
necessity in itself 

Aims & premises 

This article aims to provide a "road map" (see below) to 
assist researchers in conducting EE research, and also to 
initiate a more focused debate within bioethics about how 
to improve the quality of EE research. Our contribution 
should be understood primarily as a heuristic approach. 
As the discussion on quality criteria for EE research is 
rather new and touches on a number of complex topics 
within interdisciplinary research we would see our article 
as a first and provisional suggestion in this respect. We 
will discuss four domains of quality criteria and provide a 
tentative list of questions to be considered by researchers 
when engaging in EE research. Each formal quality criterion 
will therefore be guided by practical questions which illus- 
trate its reflective and methodological purpose. 

In this paper we will focus mainly on providing and 
discussing the abstract criteria, but will refrain from citing 
detailed examples for each criterion because of length 
limitations. While different application fields for quality 
criteria can be imagined (such as journal peer review, 
assessment of research proposals, or the planning of indi- 
vidual research projects), it should be noted that the criteria 
we present are only designed for guiding EE research (and, 
partially and indirectly, for reporting on it, since the 
reported study is what peer reviewers and readers of 
scientific literature ultimately see). 

We start from the premise - supported by our own re- 
search, and corroborated by several authors in the debate 
e.g. [6,8,14-18] - that empirical research is vital for the 
vast majority of normative ethical research. Here we focus 
on "applied ethics", that is, research concerning analyses, 
evaluations and recommendations in ethically sensitive 
fields such as medicine and clinical research, genetics and 
neuroscience, and also economics and the media. 

As far as empirical research is concerned, we limit our 
claims here to socio-empirical research, i.e. studies based 



on methodologies from the social sciences. As to the 
normative-ethical aspect of EE research, we primarily refer 
to normative-ethical research based on philosophical 
methods. While theological methods are also important 
and valuable, we did not assess them in the context of this 
work. 

Approach 

Definition of empirical ethics research 

As a descriptive definition of EE could not claim to define 
"empirical ethics" for all instances in which this term is 
used for this, (see e.g. [11,15,19]) we will confine ourselves 
to a stipulative definition covering the various ways of 
conducting EE research (e.g. [6,14,17,20,21]). 

EE research, as we understand it, is normatively oriented 
bioethical or medical ethical research that direcdy integrates 
empirical research''. Key elements of this kind of study are 
therefore that it encompasses (i) empirical research as well 
as (ii) normative argument or analysis, and (iii) attempts to 
integrate them in such a way that knowledge is produced 
which would not have been possible without combining 
them. Concerning (iii), we proceed on the assumption that 
descriptive and normative statements can and should 
be analytically distinguished from each other in order 
to evaluate their validity [22-24] . Some proponents of EE, 
e.g. those talcing a phenomenological or hermeneutical ap- 
proach (e.g. [7,25-27]), would assume that descriptive and 
normative statements are inevitably inseparable and indis- 
tinguishable. However, in the context of the current art- 
icle, we exclude from our analysis approaches to EE 
research which are mainly hermeneutically or historically 
oriented. We believe that they can fruitfully contribute to 
EE research, but these approaches are in need of specific 
quality criteria that go beyond the scope of this paper. 
Nonetheless, the development of quality criteria or 
best practice standards might also be relevant for these 
approaches. 

The above-mentioned integration of empirical research 
and a normative-ethical argument makes interdisciplinary 
work inevitable. It implies collaboration between researchers 
trained in different fields and methodologies. While it is 
theoretically possible for interdisciplinary research to be 
carried out by a single researcher skilled in more than one 
academic field, most EE research will benefit from inter- 
disciplinary research teams (e.g. [8]). This is because the 
skills needed for applying both sound empirical research 
methods and thorough normative analysis and argument 
are seldom possessed by a single researcher. 

Working in teams also offers the opportunity to over- 
come methodological biases, penchants for particular 
research approaches and intellectual myopia in terms of 
background assumptions. For example, in qualitative 
research (e.g. interviews, observations), intersubjective 
exchange during the interpretation process is a necessary 
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precondition for enhancing the validity of the results. It 
also seems unlikely, on the basis of the criteria we are 
about to present, that such interdisciplinary (team-) work 
can be done (fully) independent of other team members, 
based on a strict division of labour between empirical 
researchers and ethicists. For all these reasons, in our 
further analysis we proceed on the assumption that EE 
research should best be carried out in an interdisciplinary 
research team. 

"Road map" analogy 

We propose to use the analogy of a "road map" in order to 
structure the different criteria in our paper, applying the 
metaphor of moving through a (not yet familiar) landscape 
for the conduct of EE research. According to this metaphor, 
the following criteria can be understood as "landmarks", 
indicating what paths to take, how fast to go, and where 
to expect a rocky road or a dead end. 



about living organ donation ("bottom-up-strategy") 
regarding their use of empirical data and ethical 
concepts, and if they reflected upon that relationship; 

(ii) to study, present and critically discuss already 
established quality criteria for each of the following 
three branches of relevant criteria, viz. a) empirical/ 
social science research, b) philosophical/ 
normative-ethical research, and c) EE research 
("top-down-strategy"); 

(iii) to consider, present and critically discuss research 
ethics criteria for each of the three branches, in the 
light of our experience in EE research and 
knowledge of the EE debate; 

(iv) to develop a consensus among the authors; 

(v) to refine the different branches and reduce 
complexity for publication; and 

(vi) to draft a tentative checklist of questions which 
operationalises criteria pertinent to EE research. 



Mapping landmarks of quality and drafting a road map 

To survey specific "landmarks" of quality in EE research, 
and to draft a corresponding "road map", our procedure 
consists of the following main steps (Figure 1 Search and 
analysis strategy, provides a graphical overview of the 
search and analysis strategy the working groups used 
during the project): 

(i) to analyse selected empirical quantitative and 

qualitative studies as well as theoretical ethics studies 



Our search and analysis strategy included first (i) a 
bottom-up analysis of 10 publications, dealing explicitly 
and/or implicitly with ethical and empirical issues of living 
organ donation. This field was used as a focused case 
study to allow a comparison of quantitative (n = 3 [28-30]) 
as well as qualitative (n = 3 [31-33]) empirical studies and 
theoretical ethics publications (n = 4 [34-37]). 

This bottom-up detailed analysis revealed that firstly, 
the relevance of empirical data for ethical analysis is often 
not made explicit, secondly, empirical studies tend to be 



Step 1 



Bottom-up selective analysis of 
existing applied studies: 

Explication of relationship between empirical and 

ethical frameworks? 
Addressing quality criteria for usage of empirical 
and ethical approaches? 



step 2 



Steps 3&4 



Step 6 



V 



Comparative, en-detail analysis of 10 publications 
on living organ transplantation: 

3 quantitative studies [28-30] 

4 theoretical studies [34-37] 
3 qualitative studies [31-33] 



Major findings: 

Relevance of empirical data for ethical analysis is often not made explicit 

• Empirical studies tend to be crypto-normative in their conclusions 

• Theoretical studies refer to empirical data, but without critically reflecting r 



lethodology, or often in a positivistic manner 



Top-down selective analysis of existing literature on 
quality criteria in applied ethics / EE 



Sub-group 1: 
quality criteria in 
philosophy/normative ethics 
research 



Sub-group 2: 
quality criteria in empirical 
social science research 



Sub-group 3: 
quality criteria in EE research 



Clustering of findings in main categories (incl. mind map software) & consensus round 
along 3 standards: formal, cognitive and ethical norms (~200 criteria) 



Steps 


Focusing on criteria specific for EE (~50 criteria) along 4 major categories: 
summarize, systematize, elucidate (finding examples) 




Primary research question, 
theoretical framework & 
methodology 


Relevance 


Interdisciplinary research 
practice 


Research ethics & 
scientific ethos 



Formulating criteria into checklist questions & final consensus round (see tables 1-4) 



Figure 1 Search and analysis strategy. 
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crypto-normative in their conclusions ("crypto-normative" 
for us implies that implicit evaluations and ethical conclu- 
sions are made, but the evaluative step is not explicated), 
and thirdly, theoretical studies often refer to empirical 
data, but rarely critically reflect the empirical method- 
ology, or often tend to apply empirical data in a positivistic 
manner. 

For step (ii), we applied a more top-down strategy by 
summarising existing literature in quality criteria in the 
three relevant fields: empirical research in social sciences, 
philosophy/normative ethics research, and EE research. 
Thus, we composed three subgroups in our Empirical 
Ethics Working Group (comprising about 13 active mem- 
bers at that time; see endnote (i) for further information 
about the working group). Each group conducted a review 
of the methodological literature in the relevant area. 
We also included explicit recommendations for quality 
research drafted by scholarly societies (e.g. of psychology 
or sociology). 

Literature was searched with a narrative I selective search 
strategy (see e.g. [38]). This strategy was developed due 
to difficulties and inappropriate results when trying a 
systematic literature search by using specific search terms, 
given the interdisciplinary nature of our topic. As a conse- 
quence, we decided to broaden our approach by using lit- 
erature found via PubMed, Philpapers and Google Scholar, 
via manual search of scientific journals, as well as via expert 
opinions generated from the members of our working 
group and their connections to the respective scientific 
community. 

The subgroup of trained philosophers (MM, with non- 
authors JD and UM; see Acknowledgments) analysing the 
criteria within philosophy often had to extract informal 
and implicitly given criteria (apart from criteria directly 
related to standards of argument, e.g. logic). The principal 
subcategories of criteria for philosophical, normative- 
ethical research are the criteria of good argument (as 
discussed in informal and formal logic (see e.g. [39,40])), 
the use of specific philosophical methods (theories, ap- 
proaches) with their respective quality criteria (e.g. [41]), 
and criteria of good ethical judgement and/or decision- 
making (as discussed in models and methods of decision- 
making (e.g. [42]) (see Figure 2, Specific criteria of quality). 

The subgroup on quality criteria for empirical research 
in social sciences (LGR, GR) had to differentiate the 
literature search of journals and monographs into general 
criteria (such as adequacy of the research process, trans- 
parency, good scientific and ethical conduct) and specific 
criteria for quantitative methods (e.g. [43]), qualitative 
methods (e.g. [44-48]) or mixed-methods approaches 
(e.g. [49-51]) (see Figure 2). 

The subgroup working on quality criteria for EE re- 
search (JS, SaS) performed a selective literature review 
in relevant bioethics journals and books focusing on 



theoretical and methodological contributions to EE re- 
search. While a number of conceptual accounts to EE 
were identified, the issue of quality standards was only 
rarely addressed [52,53] . However, parts of the conceptual 
considerations as identified in the literature could be 
translated into criteria relevant to EE research [54,55]. In 
addition, the researchers drew from current EE research 
addressing 'end of life issues' which is performed in an 
interdisciplinary research group of medical ethicists with a 
disciplinary background in philosophy, sociology and 
medicine. 

Following these procedures, the findings of each sub- 
group were discussed within the whole working group. 
Our next step was to systematise the criteria identified 
by clustering them into main categories according to 
their field (empirical, philosophical, EE research), as 
well as into subcategories (e.g. different categories for 
qualitative and quantitative empirical research, or different 
categories for criteria related to logic/argumentation theory 
and philosophical approaches). The clustering was based 
on the literature search (inductive strategy), as well as on 
our own theoretical estimation of aspects relevant for 
assessing the quality of scientific work (deductive strategy). 
The summarising process was supported by mind mapping 
software to track modifications, deletions, additions or 
re-locations of criteria. The main and sub-categories in 
this mind map were generated either inductively on the 
basis of the literature or deductively by own reasoning 
against the backdrop of scientific experience and theor- 
etical knowledge. 

Finally, we derived three overarching standards of scien- 
tific research, which were subdivided into formal, cognitive 
and ethical norms (see below Three peaks that dominate 
the scenery) based especially on the philosophy of science. 
The actual quality criteria were then seen as specific ex- 
pressions of these overarching standards. 

In step (ii), issues concerning research ethics already 
found in the reviewed literature were added to the 
mind map as further criteria. Additional literature was 
also reviewed [56-59]. 

A consensus round was initiated for step (iii) where each 
criterion was again critically discussed with a view to iden- 
tify possible redundancies. Consensus was reached with 
the results of the argumentation in the discussion, which 
was most often accompanied by a final, explicit request 
if there were any dissenting votes regarding the result. 
Active members of the working group who were not 
able to participate at the consensus round (about 3 out 
of 13 members) had the opportunity to show assent or 
dissent on the basis of the sent draft of the mind map; 
there was no crucial dissent that led to a substantial revi- 
sion of the mind map. 

The next step (iv) consisted of focusing on those criteria 
that were seen as only specific to and coherent with EE 
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Quality Criteria and IVletfiods 

of Normative-Etiiicai 
(Philosopiiical & Theological) 
Research 



nformal & Formal Logic 
(„Sound Argumentation") 



Philosophical 
(or Theological) Methods 



Methods of Ethical Judgment / 
Methods of Ethical 
Decision-making 



Figure 2 Specific criteria of quality. 



Specific Quality Criteria for 
Empirical-Ethical Research 
("Empirical Ethics") 



Leading Question, 
Theoretical Framework g 
Methods 



Interdisciplinary 
Research Practice 



Research Ethics & 
Scientific Ethos 



Quality Criteria and Methods 
of Empirical (Social Science) 
Research 



Specific Standards 



Quont/rot/Ve Methods 



Qualitative Methods 



Mixed-Method Design 



research, excluding those relating to broadly empirical 
research in social sciences and philosophy. With this 
aim in mind, the working group agreed to divide specific 
EE criteria into four domains (see also Figure 1), effectively 
reducing the amount of criteria in the mind map of about 
200 (all branches, research ethics included) to about 50. In 
these four domains, the formal and cognitive norms rele- 
vant to all kinds of scientific research were specified and 
adjusted to the particular field of EE research {^'primary 
research question, theoretical framework & methods" and 
"relevance"). Furthermore, the specific interdisciplinary 
nature of EE research was addressed {"interdisciplinary 
research practice"), and issues of research ethics which 
are pertinent to EE research {"research ethics & scientific 
ethos") were considered. 

As a result, four new subgroups were established that 
had to summarise, systematise and elucidate the according 
criteria on the basis of the already found literature of 
the three aforementioned subgroups, as well as propose 
additional criteria if found necessary. The members of 
these new subgroups also became the authors of the 
paper at hand and were assigned to the four domains as 
following: a) primary research question, theoretical frame- 
work & methods (JI, SiS, SW); b) relevance (MM); c) inter- 
disciplinary research practice (JS, SaS); and d) research 
ethics & scientific ethos (LGR, GR). 

This work also led to the last step (v), the drafting of a 
refined list which allows the "road map" below to be used 
as a checklist to guide EE research. For clarity, the criteria 
included in this list are presented in tabular form. These 
tables (see below) contain each criterion, operationalised 
into questions. We decided that it was heuristically more 
effective to ask questions rather than to consider state- 
ments, and to conceive these questions as a pragmatic 
aid to guide scholars and help them reflect on their own 
research. Nevertheless, the questions that operationalise 
criteria should not be understood as simple "yes/no" quer- 
ies - instead they should function as reflective and critical 
questions designed to assess certain quality-related aspects 



of EE research. Each subgroup proposed their phrasing 
of the questions to the whole author group to achieve 
consensus on the final phrasing. 

This checklist idea is not new. It is already well estab- 
lished in other research fields, e.g. in medicine for guideline 
recommendations (GRADE [60]; SIGN [61]), quantitative 
randomised medical trials (CONSORT [62]) and observa- 
tional epidemiological trials (STROBE [63]), where they are 
used to check evidence and/or the quality of (the reporting 
of) trials. Although normative or especially ethical aspects 
are rarely explicitly mentioned in these checklists, they in- 
clude implicit normative items such as asking for ethical 
approval, informed consent, funding or possible sources 
of bias. Critical appraisal is more and more coming up on 
the agenda of evidence based medicine ([64] see also [65]). 

In analogy, we thought it necessary to render explicit eth- 
ical questions that are implicit in EE research. Looking for 
the best fitting form of presentation, our working group 
came to the consensus that we would try to adapt the check- 
list format, as we thought it will be most helpful in order 
to display the suggested criteria in a clear, feasible way. 

Discussion 

The road map 

An aerial view: spotting hills and valleys 

As is perhaps obvious, the first criteria that have to be 
considered are philosophical quality criteria, and social 
sciences quality criteria. This idea is already mentioned 
in the EE literature (e.g. [54,66]). 

However, even if (ideally) one had knowledge of both 
sets of criteria, and had the relevant skills to apply 
them, distinctive features of the quality of EE research 
would still be missed. This is due to the interdisciplinarity 
of EE research, and specifically to the complexity that the 
necessary methodological combination of the two sets of 
criteria requires. This therefore goes far beyond the need 
for cooperation between disciplines. Additional criteria 
reflect, in particular, the combination of normative-ethical 
and empirical research. 
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In the following, the "basic" philosophical and social 
science quality criteria will not again be summarised, as 
they have already been discussed in various contexts. We 
presuppose that quality work in EE research includes con- 
sideration of these already established criteria. Instead our 
major focus will be on criteria that are either (i) specific to 
EE research (i.e., not directly relevant to other ethical or 
social sciences research), or (ii) also useful in other research 
settings, but especially important for EE research. 

Three peaks that dominate the scenery: "good science" criteria 

There are three kinds of standards that dominate all re- 
search (including normative-ethical research, empirical 
research, and combinations of the two), as they are inter- 
connected by norms which are generally concerned with 
"good" scholarship. They are based on a common-sense 
definition of what good science is: formal norms (scientific 
writing), cognitive norms (general methodology) and ethical 
norms (research ethics). All quality criteria ultimately derive 
their normative power from the same general norms and 
can thus be understood (more or less) as specific expres- 
sions of these norms. 

Looldng for an interdisciplinary highway 

a) Setting up the road signs: designing a primary research 
question and selecting a theoretical framework and corre- 
sponding methods 

EE research often faces the problem that the ethical and 
empirical aspects motivating our research are intertwined 
[66] . Special attention must therefore be paid to the design 
and development of the research question. The empirical 
and normative-ethical aspects of the research question 
have to be separated clearly without being split into two 
separate research approaches [22]; at the same time, the 
relationship between them has to be elucidated [23]. This 
reflective intermediate step is relevant because it shows 
why they are part of the same research, and cannot be 
dealt with sufficiently by two (or more) separate research 
projects. The inherent link between the parts should be 
considered on several levels: the theoretical assumptions 
[67], the relevance of empirical knowledge and data to the 
ethical question and vice versa [15], the chosen method- 
ology [66,68], the type of result/data envisaged, and the 
way the result can inform further EE research, both empir- 
ical and ethical [69] (see Table 1). 

In theory-guided research, the research questions as 
well as the chosen methodology normally depend on a 
particular theoretical framework. The underlying assump- 
tions and theoretical background of both the ethical and 
the empirical parts of the research should be made explicit 
and transparent [22,67] (see Table 1). This includes theor- 
etical work on different levels. Firstly, when combining 
empirical and ethical approaches, we need to reflect on 
the compatibility of the theories used in each part. The 



combination of two theoretical approaches needs to be 
consistent. For example, the empirical discourse analysis 
of communication structures cannot simply be transformed 
into ethical questions of individual responsibility for 
decisions. A more appropriate theoretical ethical approach 
would be one that ascribes a high level of normative rele- 
vance to communication and social interrelations. An ana- 
lysis of the core concepts of the theoretical approaches 
makes it possible to test whether these approaches are 
compatible [73]. These core concepts include ideas 
such as human agency, the relationship between social 
and individual levels of agency, concepts of causality, 
the relevance and concept of gender, to name just a few 
examples [71,74]. 

Theoretical compatibility is also a point to consider when 
it comes to the question of how empirical and ethical re- 
search inform each other. Connected to this, the selection 
of both the empirical methods and the theoretical line 
of ethical argument (in short, the theoretical method) is 
crucial. Both methods need to be compatible with the 
combined theoretical framework of the research, but 
should also reflect how the results of each part can inform 
the other part [66,68]. For example - and this is far from 
being an exhaustive list -, discursive ethical approaches 
have strong links with argumentative, discursive methods 
of surveying opinion, while liberal-utilitarian ethical ap- 
proaches tend to accept opinion polls and quantitative 
surveys. Critical reflection on the chosen methods and on 
their limitations for the combined argumentation is a 
must for all EE. 

b) "Driving only when necessary": demonstrating 
relevance 

Even if an EE study is sufficiently transparent with 
regard to its primary research question and methods, it 
can still be unnecessary, or more specifically, not valuable 
[22]. From an epistemic, ethical and even economic point 
of view (due to limited research resources), one can claim 
that an interdisciplinary approach should only be favoured 
if it is likely to lead to new insights, or to broader or more 
nuanced insights than those gained from intradisciplinary 
research. One should also bear in mind that it is ethically 
problematic to expose research participants (e.g. patients 
as interviewees) to stress if the research has low relevance 
(see also section d). 

Relevance should be understood on two levels, as 
epistemic relevance on the one hand and societal relevance 
on the other. Scholars conducting EE research must be 
able to demonstrate the value of their planned project in 
at least one sense, and ideally in both (see Table 2). 

The main issue regarding epistemic relevance (e.g. 
[56-58,79]) is whether the study makes a contribution to 
academic ethics, for example by adding new knowledge 
(e.g. developing a sound ethical argumentation for the 
topic in question), contributing to an ongoing controversy 
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Table 1 Criteria related to primary research question and selecting a theoretical framework and corresponding methods 



Reflection on the relationship between empirical 
and ethical-normative/ethical-descriptive research 
questions (even if the ethical motivation is more 
prominent than the ethical research question) 



Development/use of theoretical frameworks: 



Use of empirical and ethical methods and their 
relationship to the theoretical framework: 



• Can an explicit distinction be made between the empirical and ethical research 
questions? (e.g. a distinction between interviewing patients about their wishes 
and the ethical weight given to patient autonomy) [22,23] 

• How dependent is the empirical research question on particular ethical background 
assumptions? (e.g. justification for the selection of a target group for a questionnaire: 
why do we think their opinion is ethically relevant?) [70] 

■ How is the ethical research question dependent on empirical or socio-theoretical 
background assumptions? (e.g. ethical considerations of vulnerability of a particular 
group such as pregnant women: what are the underlying anthropological or 
psychological considerations? Are there any hidden gender-related stereotypes?) [66] 

• What are the explicit and implicit research interests and motivations of the EE 
researchers? (e.g. is research with dying patients motivated by curiosity or the moral 
attempt to empower them? Is the researcher motivated to identify possible conflicts 
of interest or might the research serve mainly to produce more social acceptance of 
a technology?) [14,69] 

■ What kind of epistemic research interest motivates the researcher to combine 
ethical and empirical research? (e.g. explaining whether the aim is the evaluation 
of established ethical practice, or of measures taken to improve ethical practice; or 
whether the aim is ethical theory-building, norm-construction, or legitimization/critique 
or a particular practice) [15] 

• How can a theoretical framework be developed; what are the main limitations of 
the chosen theoretical framework? (e.g. premises and limitations of a principle of 
autonomy, when analysing macro-social interactions) [22], see contributions in [67] 

• Were potential ambiguities of central concepts considered within the theoretical 
framework? (e.g. to which extent is the concept of 'identity' used differently in current 
philosophy and in sociology when wanting to analyse the discourse of identity 
changes by neuroenhancement empirically and its ethical implications) [71,72] 

• How does the chosen medico-theoretical framework (e.g. concept of disease/health) 
fit into the ethical-normative framework? (e.g. does a science-positivistic concept of 
disease fit into a Kantian or hermeneutic approach of ethics?) [73] 

■ How does the chosen sociological-, cultural- or philosophical framework (e.g. 
concept of personal identity) fit into an ethical normative framework (e.g. approach 
to a cosmopolitical ethics of justice?) [74] 

• Are the chosen empirical methods compatible with the combined theoretica 
framework? (e.g. are interviews with doctors as experts compatible with a liberal, 
autonomy-driven approach that claims to empower patients?) 

■ What is the advantage of the chosen method in comparison to other available 
methods? (e.g. why and when to choose a deductive approach in applied ethics 
to assess ethical problems of a new technology and not an inductive, or hermeneutic 
one?) [66,68] 

• Are the chosen methodological approaches appropriate for the envisaged combined 
research question? (e.g. does the empirical method of interviewing parents generate 
results relevant for the ethical question of whether parents should be allowed to 
influence the genetic make-up of their children?) 



(e.g. providing a new perspective), or criticizing estab- 
lished positions on a theoretical or applied level. In 
other words, it needs to be clear what knowledge gains 
the research will provide in terms of the development, 
modification or application of theory. 

Another way to ensure scientific relevance, especially in 
EE research, is to develop or refine scientific methods. 
Potential for innovation (e.g. constructing a new theory 
or new instruments for ethical decision-making) should 
also be regarded as part of epistemic relevance. If there 
is a contribution to other disciplines (e.g. to social science 
debates on agency or social dependency), this also fulfils 
the criterion of epistemic relevance. Finally, the target 



group of a study, e.g. the future beneficiaries of the results 
and conclusions of this particular EE research, should be 
clearly stated. This gives guidance for research planning, 
and for the analysis of the results. 

As far as societal or practical relevance (e.g. [58,59,79]) 
is concerned, EE research should be able to show what it 
contributes to the improvement of ethical praxis. As much 
ethics research is funded by public money, it might even 
be argued that there is a moral obligation to generate not 
only scientifically valuable knowledge, but also knowledge 
that benefits society or certain groups within it, e.g. specific 
vulnerable group. This assumption relies on a normative 
understanding of ethics as a discipline aimed at providing 
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Table 2 Criteria related to relevance 

Contribution to (scholarly) ethics • Will the study possibly produce knowledge that could not be generated by relying on traditional 

(epistemic/scientific relevance): disciplinary methodologies? (e.g. overcoming too separated empirical research and separate 

philosophical discussion) [15,22] 

• Does the study aim to increase our knowledge, and if so, with regard to what? (e.g. does the study 
contribute to a balance between theoretically generated norms and empirically found norms? E.g. 
Does it revise/improve the impact of ethical guidelines?) [56-58,78] 

• Does the study aim to give input on an ongoing controversy, or does it provide a new perspective 
on it? (e.g. clarifying if relatives are able to give substitute judgment for incapacitated patients or 
not, or e.g. if post-trial access should be compelling on the basis of new evidence of consequences 
when post-trial access is not given etc.) [1 6], partly [76] 

• Does the study aim to offer substantial arguments for or criticism of established ethical positions? 
(e.g. is a contribution to theory modification or to a refinement of the application of a theory expected? 
Are descriptive presuppositions of an ethical position, such as anthropological, sociological or psychological 
assumptions, criticised?) [1,16,66] 

• Does the study aim to contribute to the development or refinement of scientific methods, especially 
methods of EE research, and if so, how? (e.g. pilot testing of a jointly developed research instrument, 
identifying the need of developing new or refined forms of interactions between researchers) [6,58] 

• Does the study aim to offer potential for innovation, and if so, what kind of innovation? (e.g. is it 
a contribution to theory-building expected? Will the study generate new instruments for ethical 
decision-making?) [5,77] 

• Does the study aim to contribute to another scientific and/or ethical discourse? (e.g. does it contribute 
to social sciences discourses?) [partly [1] 

■ Does the study clearly states to whom it is addressed, and who will benefit from its results? (e.g. 
are the addresses and/or beneficiaries physicians, nurses, social scientists, ethicists or especially 
empirical ethicists? Are policy-makers or persons in a management position addressed? [58] 

Contribution to ethical practice • Does the study aim to improve ethical decision-making? (e.g. will it produce evidence that was 

(societal/practical relevance): absent, or will it give guidance regarding the specification of accepted norms or regarding the 

interpretation of institutional or legal rules?) [6,14,75], partly [17] 

• Does the study aim to raise awareness (among actors, institutions or society) of particular ethical 
problems? (e.g. does the study identify new ethical problems, or does it highlight specific aspects 
of a known ethical problem that was not yet addressed sufficiently in practice?) [20,78] 

• Does the study aim to lead to a shift in structures and/or decision-making processes (in relevant 
institutions)? (e.g. establishing new guidelines or building new forms of committees for ethical 
review) [77], partly [23] 

• Does the study aim to establish minimum ethical standards (in relevant institutions or professions)? 
(e.g. creating new informed consent procedures for specific patient groups?) 

• Does the study aim to contribute to or stimulate public debate? (e.g. about physician-assisted suicide, 
rationing in health care, public health funds etc.) 

• Does the study aim to contribute to or stimulate a process of legislation? (e.g. proposing alteration 
of legal norms) 

• Does the study aim to articulate the need for reforms (in a certain institution or system of society)? 
(e.g. by evaluating current practices.) [6] 

• Does the study aim to articulate new ethically pertinent ecological or economic problems? (e.g. 
costs related to a broad implementation of the use of social robots in elderly care) [16] 



orientation knowledge, which is needed to detect prob- 
lems or to indirectly or direcdy improve praxis. So scholars 
should try to demonstrate the societal/practical relevance 
of their EE research whenever feasible. 

Examples of this kind of relevance could be improve- 
ments to ethical decision-making (e.g. an empirically tested 
model of ethical decision-making), raising awareness of 
ethical problems and challenges (e.g. showing that without 
regulated antimicrobial stewardship, there is a high risk of 
antibiotics increasingly losing their effectiveness), a shift in 
structures and decision-making processes (e.g. not asking 
relatives what they want, but what the patient would have 



wanted), or the establishment of minimum ethical stan- 
dards in the institutions or professions related to the praxis 
under consideration (e.g. formulation and implementation 
of guidelines). Furthermore, societal relevance is present 
when the study initiates or simply provides a stimulus for 
public debate, or leads to/assists the process of legislation. 
Articulating a need for reforms or new ethically pertinent 
ecological or economic problems - or articulating existing 
problems in a new, enlightening way - is also of societal 
relevance (e.g. [80]). 

We are not arguing that all EE research has to demon- 
strate societal or practical relevance. EE research can be 
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relevant even if it is only relevant for scientific/scholarly 
reasons. Nonetheless, as stated above, since ethical research 
seems (to us) to be ultimately guided by a practical interest 
in ameliorating (some) social practices, scholars need to 
think about potential societal relevance when planning or 
evaluating their own EE research (compare also [24]). 

c) "Enabling car sharing": guaranteeing interdisciplinary 
research practice 

Quality criteria for interdisciplinary cooperation in EE 
research encompass different stages of the research process, 
including drafting, data gathering, data analysis and con- 
clusions [36,37]. For the different stages, the following 
points are of particular importance (see Table 3). 

In the planning stage of an EE study, the reflection on 
adequate forms of interdisciplinary collaboration should 
include consideration of not only its potential benefits, 

Table 3 Criteria related to interdisciplinary research practice 



but also its limitations. Limitations are often the result of 
different professional jargons and terms [3]. It is therefore 
advisable to clarify terminology at the beginning of the 
collaboration. Furthermore, a clear distribution of compe- 
tences and responsibilities within the research team is 
needed. A further challenge in the planning of an EE study 
is posed by the literature research, which has to consider 
journals, books, and databases from diverse fields [84]. 
During data gathering, it is important to reflect the poten- 
tial bias produced by normative or descriptive assumptions 
[54]. One should also be aware of possible biases when 
selecting published empirical studies (e.g. [85]). 

During data analysis, researchers should explicitly 
discuss the relevance of ethical theories, concepts, or 
standpoints for the empirical data analysis, as well as 
possible reductionist tendencies associated with the 



Research drafting: • What form of interdisciplinary collaboration serves the needs of an EE study? (e.g. how strong and 

how often should collaborators interact? Is it necessary to have face-to-face-meetings? Who has to be 
involved in which step of the research? Is there reflection on the potentials and the limitations of the 
kind of collaboration?) [18] 

• How can the participating researchers be adequately selected? (e.g. Which disciplines/methods are 
actually needed?) 

• How can an appropriate task schedule be developed? (e.g. at which point in time is empirical data to 
be gathered) 

• Which agreements must be reached with regard to interdisciplinary communication? (e.g. consideration 
of terms used, explanation of professional jargon and development of a "common language") [3] 

• How can competencies and responsibilities be reasonably distributed within the research team? (e.g. 
despite their varying competencies, will all the interdisciplinary researchers remain actively involved in 
the research process? Who is accountable for what?) 

• How can research questions be developed jointly? (e.g. regarding different interests and disciplinary 
perspectives, or regarding the goal of the study) 

• How can the literature search be carried out? (e.g. having to acknowledge empirical-ethical studies 
from one's own thematic field as well as to acknowledge both empirical and ethical work from 
different disciplines in diverse types of publication) 

Data gathering: • How is the joint development or modification of a research instrument carried out? (e.g. Is there a 

process that allows for dissent and argument in developing or modifying a research instrument?) 

• Is there normative-ethical reflection on the empirical research process? (e.g. can implicit normativity 
be revealed that is related to a theoretical background ("social constructivism")?) 

• Is there a mutually critical appraisal by normative and empirical sciences with regard to data gathering? 
(e.g. what constitutes "good" data for the EE study) [54] 

Data analysis and conclusions: • How do normative and empirical aspects interrelate with regard to analysis and deliberation? 

° Is the analysis of the empirical data influenced by normative theories, concepts, or standpoints? (e.g. 
by a specific account of patient autonomy) 

• Is the normative deliberation influenced by the requirements of the empirical data analysis? (e.g. by 
standardization of data) [81] 

• How do normative and empirical aspects interrelate with regard to the study's conclusions? 

- Are the ethical conclusions actually linked with normative premises? (e.g. avoiding an is/ought 
fallacy) [15,23] 

° Are the empirical conclusions supported by the data, or is there a bias in the empirical results based 
on the normative conclusions? (e.g. avoiding a normativist fallacy or "wishful thinking", deducing broad 
conclusions from fine-grained data, under- or overrating of empirical data, ignoring of empirical evidence 
that would criticize normative conclusions etc.) [82,83] 

• Is there a critical evaluation of the results? (e.g. addressing methodological critique with regard to 
interdisciplinary cooperation, or indication of limitations) 



Mertz ef al. BMC Medical Ethics 201 4, 1 5:1 7 
http://www.biomedcentral.eom/1 472-6939/1 5/1 7 



Page 10 of 14 



requirements of particular methods, e.g. the standardization 
of question wording and statistical analysis in quantitative 
surveys [81]. A final crucial question is how much the 
empirical results contribute to the normative judgement 
[16,24]. Do they help to "justify" a particular norm, and 
if so, based on which ethical theoretical consideration? Or 
do they have an impact on the level of practical applica- 
tion, when public acceptance seems crucial if a rule is to 
be applied in a particular way? It is also desirable to 
consider whether the interpretation of the empirical 
data (e.g. approval or criticism of a particular group's view) 
was led by a particular ethical view [82]. 

d) "Driving responsibly": observing research ethics & 
scientific ethos 

Of course EE research is liable to general principles of 
research ethics in general and bioethics in particular. 
Important topics are avoiding harm to the participants 
[86,87], confidentiality and data safety [86,88,89], informed 
consent [86,90,91], management of research data [89,92]. 

Our discussions within our broader research group 
revealed, that in addition to that some aspects are of 
special importance in the field of EE research: 

In an interdisciplinary setting where researchers can 
have different institutional backgrounds and differ in the 
personal motivations for their research, potentially compet- 
ing interests should be disclosed and critically discussed. 
There should be explicit reflection about the interdisciplin- 
ary setting and the division of labour (see e.g. [93]). This, 
we think, must include a readiness to critically reflect upon 
a possible inclination to design the empirical part of the 
research in such a way that the results may favour the re- 
searcher's own ethical position. For example, one should 
ask: am I especially critical or joyful when it comes to this 
issue? Do I tend to exaggerate the issue in my research 
questions? Which results of the empirical research would 
I predict and which would I wish to see? Here the inter- 
disciplinary context offers an excellent opportunity for an 
open discourse (see Table 4). 

With regard to so-called informed consent, different 
standards may obtain in different disciplines such as 
medicine [94,95], psychology [96,97] or social sciences 
[98-100]. However, we think that it is the task of the 
interdisciplinary research team to openly and ethically 
consider the possibilities and to choose the most appropri- 
ate format - which may exceed the legal standard. Regard- 
less of consent by the participants, the definitive ethical 
responsibility remains with the researcher - especially as 
the EE researcher may be faced with a confidence bonus 
granted by the research participants just because she/he is 
an ethicist (an "ethical misconception" analogous to the 
"therapeutic misconception" in some clinical research). It is 
the researcher's duty to deal with this bonus very carefully. 

EE research can be misinterpreted by politicians and spe- 
cial interest groups. Therefore, we suggest that researchers 



should reflect on the following questions: have we ensured 
that the results cannot easily be misunderstood or misused? 
Might the EE study have unanticipated negative conse- 
quences that could be detected in advance and therefore 
avoided? Although it seems clear that we cannot anticipate 
every kind of negative consequences, the EE researcher 
may have a particular responsibility to carefully reflect on 
the outcomes of her/his own research beyond the short 
time frame of the study, since EE might have a strong in- 
fluence on public policies, e.g. in health care or biopolitics. 

Limitations and conclusions 
Limitations 

We assume that it is theoretically acceptable to start from 
the analytical premise that there is genuine ethical research 
on one side and genuine empirical-descriptive research on 
the other, and that these have to be paired up with each 
other through interdisciplinary research practices. We re- 
ject, on the other hand, a stance that denies the need for, 
the possibility of, or the value of strong interdisciplinary 
collaboration between empirical and ethical research (e.g. 
being afraid of naturalistic fallacies) to gain a valid applied 
ethical conclusion^. 

Our research is strongly influenced by experience and 
in-depth analysis of current EE research in the context of 
Western bioethics and medical ethics. It does not encom- 
pass other possible epistemic approaches such as revealing 
"concealed" normativity in empirical research, or more 
institutional aspects of a combination of ethical and em- 
pirical disciplines. Our "road map" is therefore restricted 
to this field. Whether this "road map" can be of use in 
other ethical disciplines, such as economic/business ethics 
or ecological ethics, has to be explored elsewhere. 

Because our literature search strategy was selective, 
there are of course limitations concerning the complete- 
ness of reviewed literature. There is a good case to believe 
that further criteria could be mentioned when including 
additional literature. But discussing quality criteria for EE 
research in a single short publication necessitates conden- 
sation and simplification; we therefore understand this 
paper as an attempt to encourage more intensive meta- 
ethical and methodological discussion within the EE 
field. 

The proposed criteria in this paper need to be tested 
in EE research practice. It is likely that the deliberate use 
of these criteria to guide and report on individual research 
will lead to the refinement, addition or removal of some 
criteria. 

Conclusions 

EE is a highly interdisciplinary and dynamic research field 
with specific methodological aspects. Because of its 
genuinely interdisciplinary nature, a reflection on meth- 
odological quality is necessarily more complex than in 
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Which personal (e.g. cultural, philosophical, theological) presumptions concerning ethics may bias 
the EE research process significantly and how can they be adequately managed? (e.g. inclination to 
a emotivist meta-ethics, a neopositivist philosophy of science, a postmodernist account of society 
etc.) [42] 

Do different standards exist in the various disciplines involved, and if so, have they been critically 
and respectfully discussed among the EE research team to find the most appropriate ethical standard? 
(e.g. is waiving of consent allowed, is assent sufficient, how to establish informed consent in emotional 
difficult situations at the end of life etc.) [43-49] 

Is the EE research team aware of a possible confidence bonus, and have strategies been developed to 
deal with this phenomenon carefully? (e.g. making transparent which goals and which limitations 
the own study will have and informing research participants and partners accordingly) 

Is the EE research team aware of the (implicit) ethical impact of the way results are presented? (e.g. 
was the potential for stigmatization or discrimination considered when choosing the wording and 
emphasis of particular results?) [30], partly [50] 

Has the EE research team made sure that the way the results are presented reduces the potential 
for misinterpretation by third parties such as politicians and special interest groups? (e.g. by changing 
perspectives when re-reading results and revising the wording etc.) 

Has the risk-benefit ratio for the EE research project been discussed in terms of its possible 
consequences for the people/society of the (near/more distant) future? (e.g. does lay considerable 
burden on study participants for a relatively low practical or epistemic output?) [30] 

Has the research team overlooked any negative consequences that could be detected in advance 
and therefore avoided? (e.g. acknowledging non-scientific partners when publishing, the handling 
of emotional distress of participants in interview studies with sensible questions, supervision of 
researchers involved in asking sensible questions etc.) 



Table 4 Criteria related to research ethics & scientific ethos 

Competing interests: 



Informed consent: 



Reporting results: 



Consequences for the future: 



traditional intradisciplinary ethical or empirical research. 
However, contributions addressing the specific challenges 
of EE research are so far rare. We have therefore tried, in 
this article, to give an overview of basic criteria which 
have to be considered to arrive at EE studies of good 
scientific quality; they may also inform the assessment 
of research protocols or proposals. We assume that 
many of the suggested criteria are self-evident, perhaps 
even common-sense - but the important thing here is the 
argument that they should be used together as a whole. 

However, our analysis also points out conflicts that 
may occur between different quality criteria, especially 
those concerning the empirical part of the research on 
the one hand and the ethical part on the other. While 
researchers may not always be able to overcome such con- 
flicts, it is paramount to at least address these different 
requirements when publishing EE research and to defend 
the chosen approach in the light of this conflict. 

Summary 

We have argued that empirical research in EE is not an 
end in itself, but vital for reaching applied normative 
conclusions. As such EE research is usually interdisciplin- 
ary, engaging in sound EE research requires more than 
merely maintaining the quality of normative argument and 
empirical method. 

Thus criteria for determining the quality of genuinely 
interdisciplinary aspects of EE research, methodological 
as well as ethical, are required. We have proposed several 
criteria of this kind under the headings Primary Research 
Question, Theoretical Framework and Methods, Relevance, 



Interdisciplinary Research Practice and Research Ethics 
and Scientific Ethos. Although these criteria cannot (yet) 
be considered definitive, they provide a starting point for 
reflection on the topic. The criteria need to be tested in 
real EE research practice and evaluation, and are likely to 
require further refinement. 

Endnotes 

■"The following paper has been jointly produced by core 
members of the Empirical Ethics Working Group (coord- 
inator until the end of 2013: Prof Dr Silke Schicktanz; 
current coordinators: Jan Schildmann and Marcel Mertz) 
of the German Academy for Ethics in Medicine (AEM; 
see http://www.aem-online.de). The working group was 
founded in 2007, with the objective of bringing together 
researchers interested in the challenges of empirical 
research in bioethics, and particularly its relevance to 
and connection with normative-ethical research. The work- 
ing group became highly interdisciplinary, consisting of 
philosophers, theologians, medical ethicists, social scien- 
tists, physicians, and humanities scholars. 
''Researchers in the field of EE use a broad variety of 
empirical research approaches, which overlap with the 
methods used in empirical disciplines such as the social 
sciences, psychology, ethnography and anthropology (see 
e.g. [16,101]). 

'^Though sometimes results from empirical research in 
bioethics, without any normative conclusions, are also 
categorised under this label (see e.g. [55,102,103]). 
''This is especially important for attempts to ethically 
improve clinical practice guidelines, ethical guidelines 
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or clinical support services by relying on empirical data, 
as the situation concerning quality criteria is similarly un- 
satisfactory (e.g. [77,104]). 

''Studies that explicitly rely on (vast) empirical research 
(results) for their ethical argumentation, but do not en- 
gage in empirical research themselves, should also be 
considered when discussing issues of quality in EE research. 
We decided, though, to focus here solely on EE studies that 
incorporate genuine empirical research. 
'Though we do not think that EE research is generally 
burdened with the problem of avoiding the naturalistic 
fallacy, or that it usually illegitimately crosses the is/ought 
gap (see [23]). 

Abbreviation 

EE: Empirical etiiics. 
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